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ABSTRACT 

We investigate the orientation correlation of giant elliptical galaxies by measuring the intrinsic 
ellipticity correlation function of 83,773 luminous red galaxies (LRGs) at redshifts 0.16 - 0.47 from 
the Sloan Digital Sky Survey. We have accurately determined the correlation up to 30 Mpc. 
Luminosity dependence of the ellipticity correlation is also detected although the error bars are large, 
while no evidence is found for its redshift evolution between z = 0.2 and z = 0.4. Then we use a 
cosmological A^-body simulation to examine misalignment between the central LRGs and their parent 
dark matter halos. Central and satellite galaxies are assigned to simulated halos by employing a 
halo occupation distribution model for the LRGs. The ellipticity correlation is predicted to have the 
same shape as but an amplitude about 4 times higher than our observation if the central LRGs are 
perfectly aligned with their host halos. This indicates that the central LRG galaxies are preferentially 
but not perfectly aligned with their host halos. With the assumption that there is a misalignment 
angle between a central LRG and its host halo which follows a Gaussian distribution with a zero mean 
and a width ag, we obtain a tight constraint on the misalignment parameter, ae = 35.4^3 3 deg. This 
type of intrinsic ellipticity correlation, if not corrected, can lead to contamination at 5% level to the 
shear power spectrum in weak lensing surveys of limiting magnitude Rab — 24.5 if the source central 
galaxies follow the same misalignment distribution as the LRGs. 



Subject headings: cosmology: observations - 
tion — galaxies: halos - 



1. INTRODUCTION 



- galaxies: elliptical and lenticular, cD — galaxies: forma- 
large-scale structure of universe — methods: statistical 



low redshift 



Weak gravitational lensing by large-scale structure pro- 
vides a unique tool that directly probes matter distribu- 
tion in the universe. One of the most serious contam- 
inations for weak lensing observations comes from two 
types of intrinsic alignments: the ellipticity correlation 
of source galaxies with each other (intrinsic ellipticity- 
intrinsic ellipticity correlation) and the ellipticity correla- 
tion of source galaxies with the surrounding matter dis- 
tribution (gravitational shear-intrinsic ellipticity corre- 
lation). 

There was much work based on both analyti- 
cal and numerical methods which attempted to es- 
timate the intrins ic ellipticity -intrinsic ellipticit y cor- 
relation f Hcavcns^ eTan 120001: ''Croft & Mctzlcr I200C " 



Brown et al.l 



en et al.ll2000l: iBernstein fc Norberg||2002l: 



- 2001 IH 

2004 iLee fc PenI l2007f r the amplitude of the cor- 
relatio n is much smaller than theoretical predic- 
tions d Hevmans et alj l2004t iMandelbaum et al.l I2006U 
iHevma ns ct al. 200;^), which indicates that galaxies are 
not perfectly aligned with the angular momentum or 
the ellipticity of their host halos. iHevmans et al.l (|2004l ) 
explained the discrepancy in amplitude of the elliptic- 
ity correlation function between the model predictions 
and the observations by assigning a random misalign- 
ment angle around the original halo angular momen - 
tum vector. On the other hand, iHoekstra et al.l (|2004l ) 
detected the significant flattening of dark matter halos 
along the minor axes of galaxies from weak lensing anal- 
ysis, which implies that the halos are well aligned with 
the major axes of the galax ies. In a later work, however, 
IMandelbaum et al] (|2006af) did not detect this effect even 
using a much larger SDSS data set. 

There are also observational studies which have 
detected with a high confidence the gravitational shear- 
intrins ic ellipticity correl ation (jMandelbaum et al.l 
I2006bl : iHirata et all l2007t ). although downweight- 
ing th is effect is mo r e complicated (iHui fc Zhand 
2008"; 'Hirata fc Seliak "2001 'Hevmans et al.' |2006!; 
Bridle fc- King 2007; Joachimi fc Schneider 2008). In 
this paper, we focus only on the intrinsic ellipticity- 
intrinsic ellipticity correlation, so we call it the intrinsic 
ellipticity correlation for brevity. 

Meanwhile, investigating intrinsic alignment of galax- 
ies relative to their host halos is also important because 
it contains information about galaxy formation and evo- 
lution. Recently there has been increasing interest in 
the misalignment between central galaxies and their par- 



irata et all 12004 iHeymans et al.l 



Lee fc Peni ,2000; Catelan et al. 12001; .Crittenden et al.l 
20011: ILee fc PenI 120011 : iJingI 120021 1. According to 
these previous studies the effect of intrinsic align- 
ment can lead to ~ 10% or even higher contamina- 
tion for a deep survey with median source redshift 
of 1 if galaxies are aligned with the angular momen- 
tum or the ellipticity of their host halos. Fortunately, 
this effect can be reduced by downweighting nearby 
source pa irs with either sp ectroscopic or photo metric 
redshifts (iKing fc Sc hneider 2002: Hevmans fc H eavens! 
I2OOI iTakada fc Whitai2004: Kins .2005 : Fan 20011) . Al- 
though the intrinsic ellipticity-intrinsic ellipticity cor- 
relation has been detected in several observations at 
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ent dark matter halos. It was shown in previous studies 
based on A^-body simulations that the angular momen- 
tum distributions of gas and dark matter components 
are partiall y aligned, with a typical misalignment an- 
gle of ^ 30° ("van den Bosch ct aL 200^ |ClieneLaI||2Q03|; 
ISharma fc Steinmetz 2005). but this might be relevant 
to disk galaxies only. It is also expected that the cen- 
tral ellipticals are aligned with their host halos to a cer- 
tain degree if the ellipticals are formed by dry mergers 
(lDubinskilli998t iNaab et al.ll2006t [Bovlan-Kolchin et al.l 
|2006[) . because the orientations of the central ellipticals 
and of the host dark matter halos are respectively de- 
termined by the orbital angular momenta of their pro- 
genitor galaxies and halos that are correlated. In ob- 
servation, by studying the alignment of central galaxies 
with their satellit e spatial distributions in SDSS groups 
()Yang et al.l l2006l ) and by assuming that t he satellites 
follow dark matter in spatial distribution, iKang et al.l 
(|2007| ) and IWang et all ()2008l ) have reached somewhat 
conflicting conclusions about the misalignment angle be- 
tween the central g alaxie s and their host halos (t ypically 
40° in IKang et all (l2007h and 23° in I Wang et all |2008)). 

In this paper we present the ellipticity correlation func- 
tions of a spectroscopic luminous red galaxy ( LRG) sam- 
ple from the Sloan Digital Sky Survey fSDSS: lYork et all 
HOOO). We estimate the luminosity and redshift depen- 
dences of the ellipticity correlations. LRGs are massive, 
and most of them are located in the central regions of rich 
groups or galaxy clusters. A small fraction of satellites 
can be reliably identified in the observation. Therefore, 
we are able to study the misalignment between central 
LRGs and their parent dark halos by comparing the ob- 
served ellipticity correlation function with that of the 
dark halos in an A^-body simulation. 

Compared to previous work concerning the misalign- 
ment, our analysis using the LRG sample have at least 
three advantages. First, the orientation of satellite galax- 
ies relative to their host halos might be very different 
from that of the central galaxies. Central LRGs are easy 
to be identified, which enables us to reliably determine 
the misalignment of central galaxies with their host halos 
without contamination from satellite galaxies. Second, 
all LRG galaxies are believed to be the product of dry 
mergers, and their formation processes are distinct from 
those of spiral (disk) galaxies. Our analysis will be on 
central ellipticals without contamination from disk galax- 
ies. Finally, because LRGs preferentially reside in mas- 
sive halos and such host halos have stronger ellipticity 
corr elations th an less massive ones hosting fainter galax- 
ies (|jiMl[20?)l ). it is easier to accurately measure their 
ellipticity correlations and determine their misalignment 
angle relative to their host halos. 

The structure of this paper is as follows. In Section 
[21 we describe the SDSS LRG sample used in our anal- 
yses. We measure the ellipticity correlation functions of 
SDSS LRGs in Section[3l The luminosity and redshift de- 
pendences of the ellipticity correlation function are also 
presented. In Section|4]we calculate the model ellipticity 
correlation functions of LRGs using an A^-body simula- 
tion with the assumption that central LRGs are com- 
pletely aligned with their parent dark matter halos. We 
examine and constrain the misalignment between central 
LRGs and their parent halos in Section [5l Our conclu- 
sions are given in Section [6l 



2. SDSS LUMINOUS RED GALAXY SAMPLE 

We analyze the LRG sample fro m the SDSS 
(jYork et al.ll2000l : IStoughto n eTaU\2m ^. The LRG se- 
lection algorithm ( jEisenstein et al selects ~ 12 
galaxies per square degree using color and magnitude 
cuts. The resulting galaxies have a Petrosian magnitude 
r < 19.5, which tend to be luminous early-types and 
to be located in rich groups or clu sters of galaxies. Al l 
fluxes are corrected for reddening (jSchlegel et al.|[T998[) 
before use. The LRG selection is so efficient that it pro- 
duces a volume-limited sample, and thus the comoving 
number density of the sample is close to a constant out 
to z ~ 0.36 an d drops thereafter due to the flux limits 
(see Figure 1 of lZehavi et 311120051 ). 

For our analysis we use 83,773 LRGs in the redshift 
range from 0.16 to 0.47 from th e SDSS Data Release 6 
(jAdelman-McCarthv et ani2008f ) which is publicly avail- 
able. We choose only the LRGs for which the redshift 
confidence parameter is greater than 0.95. The galaxies 
in the sample have rest-frame (;-band absolute magni- 
tudes -23.2 < Mg < -21.2 {Ho = 100 km s"^ Mpc"^) 
with K + E corrections of passively evolved galax- 
ies to a fiducial red shift of 0.3 (see Appendix B of 
lEisenstein et al.ll2001h . 

The goal of this study is to investigate the misalign- 
ment of central galaxies and their host dark matter ha- 
los. Accurate measurement of redshifts enables us to 
divide the LRG s in our sample i nto c entrals and satel- 
lites. Following iReid fc Spergell ()2008[ ). we adopt crite- 
ria of r±_ < 0.8 Mpc and r\\ < 20 Mpc for two 
galaxies to be in the same halo, where r± and r|| are 
their separations perpendicular and parallel to the line of 
sight, respectively. The criteria imply that 79,493 LRGs 
(94.9%) are centrals, which is consistent with the results 
not only from mock LRGs desc ribed in Sect i on 14.21 but 
also from observationa l work bv lZheng et al.l (|2008f ) and 
IReid &: Spereel (|2008D . For the halos with two or more 
LRGs, we regard the brightest one as the central and the 
others as the satellites. 

Besides a sample of LRGs, we also need information 
on their shapes. There are several model-dependent and 
model-independent ways i n the SDSS data to m easure 
the ellipticity of galaxies (Stou ghton et al.ll2002t ). Fol- 
lowing the previous wor k which investigated the central- 
satellite alignment (e.g., lBrainerdll2005l : lYang et al.l[2006l : 
iFaltenbacher et aI1l2007f l. we adopt the latter and de- 
fine the ellipticity of galaxies with the ellipticity of 
the 25 mag arcsec"^ isophote in the r band. In ad- 
dition, the point-spread function (PSF) has been cor- 
rected when m easuring galaxy shapes in the SDSS imag- 
ing pipelines (iFischer et all I2OOOI : iLupton et all 120011 : 
IStoughton et al.ll2002f ). ivfore accurate schemes for cor- 
recting t he PSF were adopt e d in previou s work of wea k 
lensing (jHirata et al.l l2004t [Mandclbau m et al.l 120051 ). 
However, the original correction for the PSF in the SDSS 
should be sufficient for the current analysis because only 
the position angles of LRGs are used in most of our study. 

3. ELLIPTICITY CORRELATION OF LRGS 

3.1. Measuring the LRG Ellipticity Correlation 
Functions 

The ellipticity correlation is examined by measuring 
the ellipticity of each galaxy. The two components of 



Misalignment of LRGs with their Host Dark Matter Halos 



3 



the elUpticity are defined as 



1 



f_ /cos 2/3 
f ^ sin 2/3 



(1) 



where q is the ratio of minor and major axes (0 < 
q < 1) and (3 is the position angle of the ellipticity 
from the north celestial pole to east (jStoughton et alJ 
I2OO2D . The n the ellipticity correla t ion functions are de- 
fined (e.g. , 'Miralda-Escude" '199lt iHeavens et all 120001 : 
ICroft fc Metz lcr 2000; Jing 20oCas 



Cab{r) = (ea(x)ef,(x + r)) 



(2) 



where r is the three-dimensional vector of separation r 
joining a pair of galaxies. The comoving distance to ev- 
ery galaxy, x{z), is calculated by assuming a flat uni- 
verse with = 1 — Oa = 0.3, where and Ha are 
the mass density parameter and the cosmological con- 
stant parameter, respectively. In Eq.([2]) the components 
of the ellipticity, ei and 62, are redefined by rotating by 
an angle between the north pole and the line connecting 
the two galaxies on the celestial sphere. Thus, the ellip- 
ticity component ei (62) corresponds to the elongation 
and compression along (at 45° from) the line joining the 
two galaxies. 

The resulting ellipticity correlation functions for the 
observed central LRGs (see Section [2]) are shown in the 
top panel of Figure [T] For comparison, the result of cn 
measured from all the LRGs including satellites is also 
given, which shows that the difference between the re- 
sults with and without satellites is very small, because 
the contribution from the satellites is negligible. The er- 
ror bars shown in the figure represent la errors estimated 
with the jackknife resampling method. A description of 
the method is given at the end of this section. 

As for the auto-correlation functions, while we clearly 
detect the positive correlation of ellipticity in cn par- 
ticularly on scales less than 30 Mpc, the amplitude 
of C22 is found to be much smaller. It is simply be- 
cause cii i s nearly isotropic but C22 is very anisotropic 
()Croft fc M etzler 2000: Jing 2002); thus the amplitude 
of C22 is suppressed when averaged over different di- 
rections. On the other hand, the cross-correlations C12 
and C21 should vanish on all scale s (jHeavens et al.ll2000l : 
iGroft fc Met^ [20001 : [jiMl[200l and our results obser- 
vationally confirm that (ci2 -I- C2i)/2 fluctuates around 
zero within the measurement errors except for a point at 
r « 4 Mpc with a 2a deviation . In the following 
analysis the function cn is mainly discussed and C22 is 
used only for a cross-check of the results obtained from 
the measurement of cn. 

The bottom panel of Figure [1] is the same as the top 
panel, except that it shows the results when the axis ra- 
tio q in equation ((!]) is set to be zero. It is equivalent 
to assuming that a galaxy is a line along its major axis, 
and the measurement indicates the correlation between 
the orientations of two galaxies with their shape not be- 
ing considered. This prescription is important because 
only information on position angles is necessary when 
we examine the misalignment between LRGs and their 
parent dark halos. Throughout this paper, we perform 
all the statistical analyses using the ellipticity correlation 
functions with q — Q. We note that this cn with q — Q 
is about 10 times larger than that with g 7^ 0, consistent 
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Fig. 1. — Top: ellipticity correlation functions for the central 
LRG sample. Bottom: as the top panel, but setting q = when 
the ellipticity of the LRGs is measured in equation jlj. The 
dashed black line shows cn for the combined sample of central 
and satellite LRGs. To clearly show the fluctuations of C22 and 
(ci2 -I- C2i)/2, mixed logarithmic and linear scalings are used 
for the vertical axis. Bins in r are in logarithmic separation of 
0.25. The circles/triangles have been respectively offset in the 
negative/positive direction for clarity. 

with the fact that the median value of q is 0.73 for the 
sample. 

Statistical errors on the measurement of the elliptic- 
ity correlation f unctions are estimated using jackknife 
resampling (e.g.. lLuptonlll993l ) with 99 angular subsam- 
ples. Because the number of data points used for our sta- 
tistical analysis is 8, this number of subsamples is large 
enough to obtain a nonsingular matrix. Each subsaniple 
includes a region contiguous on the sky, the comoving 
size of which is about 120 Mpc on a side at z = 0.3. 

We obtain the covariance matrix of cn for the central 
LRGs from the jackknifed realizations by 

N-l ^ 



1=1 



(3) 

where N = 99, c{i{ri) represents the value of cii(r) of the 
ith separation bin in the /th realization, and cii(ri) is the 
mean value of cii(rj) over all realizations. Figure [2] shows 
the obtained covariance matrix normalized by the diago- 
nal elements, Cij /{Cu-CjjY^^ . As is clearly seen, almost 
all the contribution of statistical errors comes from the 
diagonal elements. The error bars shown in Figure [T] are 



a square root of the diagonals of the matrix, C, 



1/2 
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Fig. 2. — Normalized covariance matrix for the c\\(r) 
measurement of the central LRG sample. Contour spacing is 
0.2 going from 1 on the diagonal (thick line) down to 0. The 
dashed lin e denotes the 0.4 contour. This is a similar plot to 
Figure 7 of lZehavi et al.l l|2005l ) who focused on the LRG clustering. 



have tried several values of A^, and found that the error 
bars we obtain are very stable against the changes of N . 

3.2. Luminosity and Redshift Dependences 

Before proceeding to the next section, we examine the 
dependences of the measured ellipticity correlation func- 
tion of LRGs on their luminosity and redshift. Previous 
studies fo und a luminosity dependence for the clu stering 
of LRGs (jZehavi et al.ll2005HPercival et al.ll2007f ). which 
indicates that LRGs of different luminosities are in ha- 
los of different mass. Because the ellip ticity corre lation 
of halos increases with the halo mass (|,Tingll2002D . it is 
naturally expected that there also exist the luminosity 
dependence for the ellipticity correlations of galaxies. 

Figure[3]shows cii(r) for two spans of Mg at 0.16 < z < 
0. 36. The magnitude cuts are similar to those adopted 
in IZehavi et al.l (|2005f ) and the redshift range is chosen 
to utilize a volume-limited sample. Stronger correlation 
of ellipticity can be seen in the brighter sample (—23.2 < 
Mg < —21.8) on small scales although the error bars 
are large because of the sparseness and limited survey 
volume of the sample. This result is consistent with the 
expectation that the m ore luminous LRG s are located 
m more massive halos ()Zehavi et al.l 120051: IZheng et all 
lopes') that have a stronger ellipticity correlation function 
Hing 20021. 

OLee fc PenI ()2008l ) analyzed simulation data in order to 
pursue the nonlinear evolution of galaxy intrinsic align- 
ment at < z < 2. They did not find an indication 
for the redshift evolution of the ellipticity correlation al- 
though they could detect the evolution of its nonlinearity. 
Thus it may be difhcult for current observations to de- 
tect the redshift dependence of the ellipticity correlation. 
Redshift evolution for the L RG clustering was also not 
detected (jZehavi et al.ll2005l ). Figure U) shows the red- 
shift dependence of cii(r). Because the redshift range of 
our LRG sa mple is not l arge, even shallower than that 
examined in iLee fc PenI (|2008D . we do not find such a 
dependence in the LRG sample. 
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Fig. 3. — Luminosity dependence of cii(r) of the LRGs for 
0.16 < z < 0.36. Here both the central and satellite LRGs are used 
for the calculation. Note that the vertical axis mixes logarithmic 
and linear scalings. 
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Fig. 4. — Redshift dependence of cii(r) of the LRGs for the 
absolute magnitude of -23.2 < Mg < -21.2. Here both the 
central and satellite LRGs are used for the calculation. Note that 
the vertical axis mixes logarithmic and linear scalings. 



4. MODEL PREDICTIONS 

4.1. N-body Simulation 

To make model predictions for the ellipticity correla- 
tion function, we use a halo catalog constructed from a 
high-resolution cosmological simulation with 1024*^ par- 
ticles in a cubic box of side 1200 h^^ Mpc (|Jing et al.l 
I2007t l. A spatially flat ACDM model with = I-Qa = 
0.268, ilb = 0.045, h = 0.71, and as — 0.85, was assumed, 
where fli, is the baryon density parameter, h is the Hub- 
ble parameter normalized by 100 km s~^Mpc~^ , and erg is 
the present linear rms density fluctuation within a sphere 
of radius 8 h^^ Mpc. Dark matter halos are identified 
in the z = 0.274 output using the friends-of-friends algo- 
rithm with a linking leng th b equal t o 0.2 times the mean 
particle separation. See iJing et aTl ([2007,) for details of 
the simulation. 

4.2. Halo Occupation Distribution 

In order to assign galaxies to the simulated halos, 
we rely on the frame work of the halo occupation dis- 
tribution (HOD, e.g.. iJing et al] 119981 : [Ma fc Frvl 120001 : 
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Peacock fc SmithI l2000t ISeliakI l2000t IScoccimarro et all 
200H iBerhnd fc Weinberd l2002l ). which describes the 



relationship between the galaxy and dark matter den- 
sity fields. HOD modeling has been performed 



(iMasiedi et al. 


120061 iHo et all l2007l: iBlake et al. 


2008 


Ktdkarni et al. 


120071 iWhite et al.l 


20071 ISeo et al. 


2008 


Zheng et al.ll2008l: iReid & Soergell 


20081). 



A flexible parameteriz ation with fi v e HO D param- 
eters was introdu ced by IZheng et al.l (|2005f ) (see also 
IZheng et al.l l2007l ). The mean occupation function of 
galaxies within a dark halo of mass M, being the sum 
of central and satellite mean occupation functions, is pa- 
rameterized as 



{N{M)) = (iVcc„(Af)) (1 + (iVsat(M))) , 



(4) 



(7V,at(M)) : 



M- 



- erf 



log M - log M„ 



where erf is the error function, A/min is the characteristic 
minimum mass to host a central galaxy, Af( is a mass for 
a halo with a central galaxy to host one satellite when 
Mq <C Af{, Mo is the mass scale to truncate satellites, 
and om is the characteristic transition width. Following 
the latest fits fo r the HOD parameters of the LRGs by 
ISeo et all (|2008f ). we choose M^in = 8.226 x 10"Mq, 
M{ = 6.875 X lO^^'Mo, Mq = 3.209 x IO^Mq, om = 
0.556, and a = 1.86. A central LRG is assigned to a halo 
based on the nearest integer distribution with the average 
of {Nccn{M)). Satellite LRGs are then assigned to each 
halo with a central based on the Poisson distribution 
with the average of {NgutiM)) . The satellite LRGs inside 
dark matter halos ar e distributed followi ng the Navarro- 
Frenk- White profile (jNavarro et al.llT997T ). The resulting 
fraction of central LRGs is 93.7%, consistent with that 
from the observation (Section [2|). 

In Figure [5l we show a comparison of the real-space 
correlation fu nction between the mock and observed 
(jZehavi et al.|[2005.) LRGs. Very good agreement of the 
results between the observation and mock catalog can 
be seen except for r < 0.5 Mpc, as was seen by 
ISeo et all (|2008[ ). This small discrepancy is irrelevant 
to the current study because the satellite distribution 
within halos dominates on this scale and only central 
LRGs are used for the statistical analysis below. 

4.3. Modeled EUipticity Correlation Function 

The principal axes of each halo in a projected plane 
are comput ed by diagonalizing the momentum of inertia l 
tensor l'e.g.. lMiralda-Escudell991l : lCroft k Met^[20Ml 



E 



(5) 



where the sum is over all the particles in the halo. The 
ellipticity components of each halo are then estimated in 
the same way as those of LRGs (eq. [Tj ) , where the value 
of q is assumed to be zero again. 

First we assume that all central galaxies are completely 
aligned with their parent dark matter halos. Then the el- 
lipticity correlation functions of central galaxies are equal 
to those of their parent halos. With this assumption, 
we plot the ellipticity auto-correlation functions of the 
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Fig. 5. — Comparison of the real space correlation functions 
between the observed and mock LRGs. The black po ints with the 
error bars show the observed correlation function (IZehavi et al.l 
120051) . The dashed line is that of the mock galaxy catalog using 
the best-fit HOD model for the LRGs (Seo ct al. 2008;). 

mock LRGs, cn and C22, in Figure [6l In order to refine 
the statistics, we averaged over seven mock LRG samples 
with different random seeds for assigning LRGs to dark 
halos. Interestingly, the ellipticity correlation function 
cii of the mock LRGs has a very similar shape to the 
observed function, but the amplitude is about 4 times 
higher. The function C22 is significantly negative at r 
about a few Mpc, compared to the real observed 
one. In the next section we will explain these differences 
between the observation and simulation by considering 
misalignment of central galaxies with their host halos. 
In Figure [6] we also show the angular separations with 
the assumption of all the galaxies being at z = 1, which 
is the typical redshift of recent weak lensing surveys (see 
Section [6]). Note that the values of the ellipticity corre- 
lation function of halos are about an order of m agnitude 
larger than the previous result bv I Jind ()2002l ). because 
we assume g = in the current study. 

5. CONSTRAINTS ON MISALIGNMENT 

In this section we consider a more general case in which 
the position angle of each central galaxy is not completely 
aligned with its host halo. We assume that the proba- 
bility distribution function (PDF) of the misalignment 
angle 9 between the major axes of central LRGs and 
their host halos is a Gaussian function with a zero mean 
and a width ag , 



f{0;ag)d0 



1 



2TTae 



exp 



de. 



(6) 



where ae is the misalignment angle parameter or the 
typical misalignment angle. We artificially assign mis- 
alignment to position angles of each mock central LRG 
according to equation([6]) before the ellipticity correlation 
function is measured. It is expected that the larger the 
value of ae is, the smaller the amplitude of the ellip- 
ticity correlation function systematically becomes. For 
each chosen value of ag and each LRG mock sample, 
we generate nine misaligned LRG samples by choosing 
different random seeds in order to refine the statistics 
of our model predictions. Although the obtained nine 
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Fig. 6. — EUipticity auto-correlation functions of the central 
LRGs, (top) cii(r) and (bottom) C22(f)- In both panels, the data 
points with the error bars are the measurements from the SDSS, 
the same ones as those in the bottom panel of Figure [T] The 
dashed red lines are results of the mock central LRGs with no 
misalignment with their parent halos. The solid red lines are those 
with the misalignment parameter of ag = 35°. The horizontal 
axis at the top shows the corresponding angular scale when all the 
galaxies are located at 2 = 1. 

mock ellipticity catalogs are not independent each other, 
their average can reduce random fluctuation from differ- 
ent random seeds. Finally, our model prediction for each 
misalignment parameter ag is calculated by averaging 
over 7 X 9 = 63 misaligned samples. 

In comparing the observational data with the model 
prediction, we first compute the model ellipticity corre- 
lation function with a given parameter of ag to be tested, 
statistics are then calculated as 

where Cij is the covariance matrix given by equation 
([3]), Acii{ri; ae) the difference between the observed and 
the model values in the zth separation bin, and i and 
j runs over the number of bins. In this analysis the 
number of bins is 8 and the degree of freedom is 7. Thus 
the 99 realizations constructed from jackknife resampling 
are large enough to derive a nonsingular matrix, as was 
already stated in Section 13.11 The range of ag in our 
calculation of is 20 < ag < 50° with the width of 
Aag — 1°. The binned values of are then cubic-spline 
interpolated. 
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Fig. 7. — distribution for misalignment angle parameter ag. 
The best-fit parameter is ag = 35.4^3 3 deg (68% C.L.) when we 
use the full covariance matrix, while ag = 35.0^3 g deg when we 
use only the diagonal elements of the covariance. The minimum 
value of is X^in ~ 3.983 and 2.915 with 7 dof, respectively. 
The horizontal dotted lines show 68%, 95%, and 99% confidence 
levels. 



Figure [7] shows a function of the misalignment 

parameter ag. For comparison, both the results that all 
the elements and only diagonals of the covariance ma- 
trix are used are given. These two results are in very 
good agreement, indicating that the non-diagonal ele- 
ments of the error matrix are not important. The fits of 
the observed ellipticity correlation function to the model 
prediction using the full covariances give ag — 35. 'it.'^ '^ 
(68% confidence level), and Xmin = 3-983 with 7 dof. 

The model prediction of cn with ag = 35° is shown in 
the top panel of Figure [H] As a cross-check, we also plot 
the model of C22 with the same ag in the bottom panel 
of Figure [6l which is also in very good agreement with 
the observed C22 • This accordance additionally enhances 
the validity of our analysis. 

R ecently there were t wo papers bv iKang et al.l ()2007l ) 
and I Wang et al.l (|2008l ) who studied the misalignment 
angle between galaxies and their host halos. Although 
they used the same observed statistics of the alignment 
angle between the major axis of th e central g alaxies 
and their connecting lines to satellites ljYang et al.| [2b06). 
they obtained the typical misalignment angle with dif- 
fer ent results (a bout 40° by Kang et al. (2007) and 23° 
by I Wang et al.l (|200 8) for the whole sample of blue and 
red central galaxies in their papers). The difference may 
come from their different methods to trac e the satel- 
lite spatial distribution in their modeling. IKang et al.l 
(2007) h ave used a semi-an alytical model to trace satel- 
lites, and lWang et al.l (|2008D have first tried to determine 
the spatial distribution of satellites within halos. The 
discrepancy might come from the fact that the triaxial 
sh ape of satelli t e dist ribution within groups determined 
by iWang et"al] (|2008l ) is much rounder than dark mat- 
ter h alos in simulations (jJing fc Sutol 120021 : IKang et al.l 
|2007( ). Our analysis does not need to make any assump- 
tion for the satellite galaxies or the shape of halos. Be- 
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cause LRGs are red centrals, our results should be com- 
pared with the misalignment valu e 16.6 ±0.1 degrees for 
red centrals in lWang et all (l2008|) (their Table 2). Their 
value is significantly smaller than ours, and such a small 
value is strongly rejected by our analysis. 

6. CONCLUSIONS 

We have measured the ellipticity auto- and cross- 
correlation functions Ca(,(r) of the spectroscopic LRG 
sample from the SDSS. We have detected positive align- 
ment between pairs of the LRGs up to ~ 30 h'^ Mpc 
scales. More luminous LRGs have a stronger correlation 
of ellipticity than less luminous ones although the error 
bars are large, while no significant evidence is found for 
redshift dependence between redshifts 0.2 and 0.4. 

Accurate measurement of spectroscopic redshifts en- 
ables us to divide our LRG sample into centrals and 
satellites. In order to examine the existence of misalign- 
ment between central LRGs and their host dark matter 
halos, we employed a high-resolution iV-body simulation. 
Adopting the best-fit HOD parameters of the LRGs al- 
ready obtained in previous studies, we assigned central 
and satellite LRGs to simulated halos. Then we mea- 
sured the model ellipticity correlation functions from the 
mock central LRGs. The ellipticity correlation is pre- 
dicted to have the same shape as, but an amplitude about 
4 times higher than, our observation if the central LRGs 
are perfectly aligned with their host halos. 

We assumed misalignment of the central LRGs with 
their parent dark halos to follow a Gaussian distribution 
with zero mean and a width ag. By comparing the ob- 
served ellipticity correlation function cn with its model 
predictions, we have obtained a tight constraint on the 
misalignment parameter as ae = 35.4t^:°. A model that 
the LRGs and host halos are completely aligned was 
strongly rejected in our analysis. This is an accurate 
detection of the misalignment using observed data. 

The results have important implications for weak lens- 
ing observations. To be specific for the current discus- 
sion, we assume that the source galaxies are at redshift 
about 1 and have limiting magnitude Rab — 24.5 similar 
to that of a re cent large GFHTLS weak len sing survey 
(iFu et al.ll2008[ ). From the HOD analysis by Zhen g et al.] 
(120071 ^ for the clustering of galaxies in the DEEP2 red- 
shift survey which has a similar limiting magnitude, we 
know that more than ~ 80% of these source galaxies are 
central galaxies in dark matter halos ~ 4 x IO^^H'^Mq. 
If these central galaxies have the same misalignment dis- 
tribution relative to their host halos as the LRGs, we ex- 
pect that the intrinsic ellipticity correlation of the galax- 
ies is about 4 times smaller than that of the hosting halos. 
Considering that the distribution of the intrinsic elliptic- 
ity of the galaxies at z « 1 is quite similar to that of host 
halos (G. Gao et al. 2009, in preparation), we expect 
that this intrinsic ellipticity correlation of galaxies can 
contribute by about 5% to the shear correlation accord- 
ing to t he elliptici ty correlation function of halos (Fig- 
ure 3 of ljindl2002D and the shear power spectrum (e.g., 
IWittman et ani2000t lKaiseill2000D . The contribution is 
decreased if the blue disk galaxies are misaligne d more 
with their host halos than the red ones (see, Yan g et aTl 
|2006[ ). A more precise assessment of the impact on the 
weak lensing observations will be discussed in a future 



paper based on more detailed modeling of galaxies in 
halos. 

The misalignment distribution of the LRGs can also 
serve as a test for the formation of giant ellipticals. The 
dry and gas-rich mergers may result in different prop- 
erties of elliptical galaxies, including their orientations. 
This may also serve a test for the hierarchical formation 
scenario of galaxies. We will compare this observation 
with the central galaxies in a hydro/ A'^-body cosmologi- 
cal simulation in a future paper. 

The current SDSS-II Legacy Survey will be completed 
in the next Data Release. However, an upcoming survey, 
the SDSS-IH's Baryon Oscillation Spectroscopic Survey 
(BOSS)^, is planned to intensively survey the LRGs at 
the more distant universe, 0.5 < z < 0.8. Although 
the main goal of the BOSS is accurate measurement of 
cosmological distance scales using b aryon acoustic oscil- 
lations (e.g.. [ETsenstein et al.|[2005[) . it will also enable 
us to discuss the evolutionary effects of LRGs. When 
LRGs at wider redshift ranges are available, we will be 
able to improve the results obtained in this work. Then 
the luminosity and redshift dependences of misalignment 
between central galaxies and their host halos will be dis- 
cussed more accurately, which will lead to a better un- 
derstanding of galaxy formation and evolution. 
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